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Key findings 


This study, conducted in the Cleveland Metropolitan School District, explored the 
relationships between student and school characteristics among English learner 
students and their performance on math, English language arts, and English language 
proficiency assessments. The key findings include: 
Four school climate domains—academic rigor, safe and respectful climate, peer 
social-emotional learning, and supportive learning environment—were associated 
with higher English language speaking proficiency levels. 
Two school climate domains—academic rigor and supportive learning environnent— 
were associated with higher English language arts performance. 
A higher number of students per bilingual paraprofessional was associated with 
lower math performance. 
A higher number of students per certified teacher of English as a second language 
was associated with lower English language speaking proficiency levels. 
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Summary 


The Cleveland Metropolitan School District has seen the number of English learner stu- 
dents in grades K-12 increase in recent years, even as overall enrollment has decreased 
(U.S. Department of Education, National Center for Education Statistics, 2016). The pro- 
portion of students in the district classified as English learner students rose from 5.4 percent 
in 2011/12 to 8.8 percent in 2016/17. In addition, the English learner student population 
has grown more diverse in race/ethnicity, country of origin, and native language. This has 
been driven in part by an influx of resettled refugees speaking a variety of languages. The 
increasingly diverse English learner student population requires more support from the dis- 
trict to meet broader needs for language, cultural, and educational assistance. 


The Cleveland Partnership for English Learner Success is a researcher—practitioner part- 
nership between the Regional Educational Laboratory Midwest and the Cleveland Met- 
ropolitan School District’s Multilingual Multicultural Education Office and the district’s 
research office. The partnership has prioritized examining the characteristics of English 
learner students and the schools they attend to identify which characteristics are associat 
ed with student academic performance and English language proficiency. The study team 
conducted this research for the partnership as a step toward improving district and school 
support for English learner students. 


The study described the characteristics of English learner students in grades 3-8 and the 
schools they attended, as well as the students’ performance on statewide math, English 
language arts, and English language proficiency assessments in 2011/12—2016/17. It then 
examined the relationships between those characteristics and student academic perfor 
mance and English language proficiency. The key findings include: 

e Four school climate domains—academic rigor, safe and respectful climate, peer 
social-emotional learning, and supportive learning environment—were associated 
with higher English language speaking proficiency levels. 

e Two school climate domains—academic rigor and supportive learning 
environment—were associated with higher English language arts performance. 

e A higher number of students per bilingual paraprofessional was associated with 
lower math performance. 

e A higher number of students per certified teacher of English as a second language 
was associated with lower English speaking proficiency levels. 


While this study cannot determine whether the relationships are only correlational, the 
findings are a first step toward identifying action areas that the district can consider for 
supporting English learner students. The findings also may be relevant for other urban 
districts with smaller but growing English learner student populations. 
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Why this study? 


Like many large and midsize districts in the Regional Educational Laboratory (REL) 
Midwest Region, the Cleveland Metropolitan School District (CMSD) has seen its English 
learner student population in grades K-12 increase in recent years (U.S. Department of 
Education, National Center for Education Statistics, 2016).! Specifically, from 2011/12 to 
2016/17 the number of English learner students in the district rose 27 percent, while overall 
enrollment decreased 11 percent. Thus, the proportion of English learner students in the 
district rose from 5.4 percent in 2011/12 to 8.8 percent in 2016/17. Effectively supporting 
the academic performance and English language development of the growing population 
of English learner students is a key priority for the district and has served as the basis for 
the Cleveland Partnership for English Learner Success between REL Midwest and CMSD. 


The challenge that CMSD faces extends beyond the increase in English learner students; 
the composition of the English learner student population is diversifying in terms of race/ 
ethnicity, country of origin, and native language. From 2011/12 to 2016/17 the percent 
age of non-Spanish-speaking English learner students nearly doubled in the district, from 
18 percent to 30 percent. These changes require the district to provide more support to 
meet broader needs for language, culture, and educational assistance. To help inform its 
allocation of resources, the district thus sought information on whether diverse student 
characteristics are associated with academic performance and English language proficiency. 


CMSD also has a particular interest in better understanding the characteristics of the 
schools that English learner students attend (see box 1), the changes in those charac- 
teristics over time, and their association with English learner students’ academic perfor 
mance and English language proficiency. Understanding the relationship between school 
characteristics and English learner student outcomes can help the district identify areas 
on which to focus. For example, the district has prioritized understanding whether the 
number of English learner students per certified teacher of English as a second language or 
per bilingual paraprofessional is associated with student outcomes. Information on school 
specialist staffing and training background can help guide the conversation about staffing 
resources and pinpoint areas for future study. (For previous research on school characteris- 
tics associated with English learner student outcomes, see appendix A.) 


The district also was interested in how the proportion of English learner students at a 
school is associated with student outcomes. Students entering CMSD who speak a primary 
language other than English are referred to the district’s Multilingual Multicultural Edu- 
cation Office before they are enrolled in a school. The office generally advises families 
to select a school in which there is a large proportion of other English learner students, 
because those schools offer more language support and provide students a peer group with 
some commonalities. While the study findings cannot directly inform school placement 
decisions, the district sought information on whether the proportion of English learner 
students at a school merits further investigation as a factor in student outcomes. 


Finally, CMSD strongly emphasizes addressing challenges in school climate, which has 
resulted in the district administering the Conditions for Learning Survey (Osher, Kendzio- 
ra, & Chinen, 2008) to students since 2008/09. Although school climate is an area of 
interest for the district, little analysis had been conducted to understand the association 
of school climate with English learner student outcomes. One goal of the current study is 
to provide a better understanding of the associations between school climate and English 
learner students’ academic performance and English language proficiency. 
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Box 1. School options for English learner students in the Cleveland Metropolitan 
School District 


The Cleveland Metropolitan School District has a school-of-choice policy that allows students 
to attend any school in the district regardless of location or distance from their home. English 
learner students in grades K-12 in the district can attend either bilingual schools,+ which 
provide more intensive support for both student native language and instruction in English, 
or standard neighborhood nonmultilingual schools, which offer varying levels of support for 
instruction in English. In addition, students whose families have lived in the United States less 
than one year are strongly encouraged (but not required) to attend the district’s newcomer 
academy for one to two years. The three types of schools offered by the district are described 
below. 


Bilingual/dual language schools. The district has seven bilingual/dual language schools 
serving grades K-8. Students can receive primary language instruction for multiple languages 
besides English and English as a second language in a self-contained classroom, or two-way 
immersion Spanish-English instruction. 


Nonmultilingual schools. The district has 67 nonmultilingual schools serving grades K-8. 
These schools offer varying levels of English as a second language and sheltered English 
instruction using the research-based Sheltered Instruction Observation Protocol model. This 
model calls for integrating English learner students with native English speakers. Certified 
English as a second language teachers use curricula and methods designed to promote 
academic content and English language skills. Not all nonmultilingual schools enroll English 
learner students. 


Newcomer academy. The district has one newcomer academy serving English learner stu- 
dents in grades K-12. The academy provides sheltered English instruction using the Sheltered 
Instruction Observation Protocol model, along with bilingual support designed specifically for 
students who are new to the country and at the beginning English language proficiency level. 
Students attend the academy for one to two years and receive the most intensive supports to 
help them transition to other district schools. Examples of intensive supports include five daily 
instructional periods of sheltered English instruction and individualized support for refugees 
and students who have experienced interruptions in schooling. The length of enrollment differs 
by student and is based on the level of support needed. Students are then encouraged to 
attend a bilingual school to continue to receive a high level of support. 

Note 

1. The bilingual curriculum is generally Spanish-English. 


Source: All definitions of language support and school types were provided by the Multilingual Multicultural 
Education Office, http://www.clevelandmetroschools.org/Domain/43; retrieved February 8, 2018. 


What the study examined 


This study answers three research questions about English learner students in grades 3-8 
in CMSD: 


1. What were the characteristics of English learner students and the schools they attend- 
ed between 2011/12 and 2016/17? 


2. What were the patterns of performance on statewide math, English language arts, 


and English language proficiency assessments among English learner students between 
2011/12 and 2016/17? 


3. Which student and school characteristics were associated with math performance, 
English language arts performance, and English language proficiency level in 2016/17, 
after other student and school characteristics are accounted for? 


Student and school characteristics were examined separately for each year, enabling the 
study team to identify patterns of stability and change. To explore associations with aca- 
demic performance and English language proficiency, the study team used data for the 
most recent year for which data were available (2016/17) in order to provide information 
that was most relevant to the current English learner student population and education 
setting. An overview of the study’s sample, measures, and methodology is presented in box 
2 (see appendix B for further details). 


Box 2. Samples, measures, and methodology 


Sample 
The sample for research questions 1 and 2 consisted of all English learner students in grades 
3-8 in the Cleveland Metropolitan School District from 2011/12 to 2016/17, and the sample for 
research question 3 consisted of the 2016/17 cohort of English learner students in grades 3-8, the 
most recent cohort for which data were available. The study focused on grades 3-8 because these 
are the grade levels that participate in statewide math and English language arts assessments. 
The district assesses the English language proficiency of all English learner students each 
year, so any student who took the English language proficiency assessment in a given year 
was identified for the study sample as an English learner student, regardless of how he or she 
scored. Schools were classified as enrolling an English learner student if they enrolled one or 
more such students in a given school year. 


Student outcomes 

The study examined student math and English language arts performance on statewide 
assessments administered each spring. Scores were standardized to the district average in 
each grade level for each school year, so that each student’s score represents his or her per- 
formance relative to all students in the district. Ohio used three standardized assessments of 
math and English language arts during the study period: the Ohio Achievement Assessment in 
2012-14, the Partnership for Assessment of Readiness for College and Careers in 2015, and 
the Ohio State Test in 2016 and 2017. 

The study also examined student performance on the state English language proficiency 
assessment, including overall proficiency levels and proficiency levels on the assessment’s four 
subscales—listening, speaking, reading, and writing. Ohio used two assessments of English 
language proficiency during the study period: the Ohio Test of English Language Acquisition in 
2011/12-2014/15 and the Ohio English Language Proficiency Assessment in 2015/16—2016/17. 


Student characteristics 

Student characteristics included prior-year English language proficiency levels, native language, 
race/ethnicity, gender, special education status, gifted status, grade level, and prior-year math 
and English language arts performance. 


(continued) 


Box 2. Samples, measures, and methodology (continued) 


Student native languages were categorized into five categories: Spanish; Arabic; African 
language (Bassa, Dinka, Ewe, Igbo, Karen, Kinyamulenge, Kirundi, Krahn, Maay Maay, Somali, 
Swahili, Tigrinya, and Yoruba); Asian language (Burmese, Cambodian, Cantonese, Filipino, 
Hindi, Indonesian, Mandarin, Nepali, Punjabi, Tagalog, Tamil, Thai, Urdu, and Vietnamese); and 
European/other language (Albanian, Bosnian, Croatian, English/Creole, French, Romanian, 
Russian, Turkish, and Ukrainian). 

For details on student characteristics, see appendix B. 


School characteristics 

School characteristics included school type, number and percentage of English learner stu- 
dents, number of languages spoken among English learner students, percentage of English 
learner students whose native language is Spanish, percentage of English learner students 
whose native language is Arabic, school specialist staffing (number of English learner students 
per certified English as a second language teacher and number per bilingual paraprofession- 
al), school climate measures, percentage of English learner students at each English lan- 
guage proficiency level, percentage of all students who met proficiency standards in math and 
English language arts, school size, percentage of all students in each racial/ethnic category, 
percentage of all students with special education status, and percentage of all students with 
gifted status. 

School climate was measured by school-level average scale scores for each of the four 
Conditions for Learning Survey domains—perceptions of academic rigor, safe and respect- 
ful climate, peer social-emotional learning, and supportive learning environment—for all years 
except 2011/12, when the survey used a different scoring method. 

For details on school characteristics, see appendix B. 


Methodology 

To address research questions 1 and 2, the study team calculated student- and school-level 
averages and percentages for each school year in the analysis. For school characteristics, 
descriptive statistics are based on all students in the school, regardless of their English 
learner status; however, only schools that enrolled English learner students are included in the 
analyses. 

To address research question 3, the study team conducted a series of hierarchical linear 
models with students grouped within schools to estimate the relationship between student 
and school characteristics and each student outcome. The analyses accounted for the student 
and school characteristics described earlier and focused on the most recent cohort of stu- 
dents in grades 3-8 enrolled in 2016/17. 


Note 


1. Bilingual paraprofessionals, also referred to in the district as “bilingual instructional aides,” take on several 
roles, which can include working directly with English learner students, providing support for instructional activ- 
ities for a primary classroom teacher, and participating in instructional planning. 


What the study found 


This section details nine notable findings for the three research questions addressed in the 
study. (For full results, see tables CI-C16 in appendix C. For an overall summary of the 
findings on the relationships between student and school characteristics and English learner 
student performance and English language proficiency levels in 2016/17, see table C15.) 


The number of English learner students speaking languages other than Spanish increased 


The number of English learner students in grades 3-8 in CMSD increased from 1,266 in 
2011/12 to 1,542 in 2016/17, a 22 percent increase. The native languages most common 
among these students were Spanish, Arabic, African languages, and Asian languages. 
From 2011/12 to 2016/17 the percentage of English learner students whose native language 
was Spanish decreased from 82 percent to 71 percent (figure 1; see also table Cl in appen- 
dix C). During the same period the percentage whose native language was Arabic more 
than doubled, from 4 percent to 9 percent, and the percentage whose native language was 
an African language doubled, from 5 percent to 10 percent. 


The percentage of English learner students attending the district’s newcomer academy increased, 
while the percentage of English learner students attending bilingual schools decreased 


The percentage of English learner students in grades 3—8 attending the district’s newcomer 
academy more than doubled from 11 percent in 2011/12 to 23 percent in 2016/17 (figure 
2). During the same period the percentage attending one of the district’s seven bilingual 
schools decreased from 70 percent to 54 percent. While there were some fluctuations from 
year to year, the percentage attending a nonmultilingual school remained generally stable. 
English learner students attended 41-47 K-8 schools each year of the study (see table C2 in 
appendix C), which represented, on average, 56 percent of the K-8 schools in the district. 


Figure 1. The percentage of English learner students in grades 3-8 who spoke 
Spanish decreased between 2011/12 and 2016/17, while the percentages who 
spoke Arabic or African languages increased 
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Note: See box 2 for the languages included in each category. 


Source: Authors’ calculations based on data from the Cleveland Metropolitan School District. 


Figure 2. The percentage of English learner students in grades 3-8 attending the 
Cleveland Metropolitan School District’s newcomer academy more than doubled 
from 2011/12 to 2016/17 
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Note: See box 1 for descriptions of each school type and the number of schools in each category. 


Source: Authors’ calculations based on data from the Cleveland Metropolitan School District. 


English learner students increasingly attended schools with school climate scores higher than the 
district average 


Between 2012/13 and 2016/17 English learner students in grades 3—8 increasingly attend- 
ed schools that scored above the district average in each of the four domains of the 
Conditions for Learning Survey: academic rigor, safe and respectful climate, peer social- 
emotional learning, and supportive learning environment (figure 3; see also table C2 in 
appendix C). For example, the average English learner student attended a school with an 
academic rigor score 0.16 standard deviation above the district average in 2012/13 and 0.73 
standard deviation above the district average in 2016/17. 


This increase occurred at the same time that more students enrolled in the newcomer 
academy. In every year during 2012/13—2016/17 the newcomer academy had school climate 
scores at least one standard deviation above the district average on most school climate 
domains (see table C5 in appendix C). As the proportion of English learner students 
attending the newcomer academy increased, the average English learner student experi- 
enced a more positive school climate. 


English learner students’ English language arts performance was below the district average, while 
their math performance was closer to that of other students 


English learner students in grades 3-8 performed below the district average on the state 
assessment in English language arts in 2011/12—2016/17 and on the state assessment in 
math in 2011/12, 2012/13, 2013/14, and 2016/17 (figure 4; see also table C6 in appendix C). 
But their math performance was closer to the district average than their English language 
arts performance was. For both subjects the degree to which English learner students 
scored below the district average fluctuated. The fluctuations coincided with the changes 
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Figure 3. English learner students in grades 3-8 attended schools that scored 
above the district average on all four domains of school climate from 2012/13 to 
2016/17 
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Note: The figure is based on school-level scores weighted by the number of English learner students per 
school to depict the average English learner student experience. Each point represents the average school 
climate score in standard deviations from the district average among English learner students in a domain and 
year. The study team standardized scores for each domain relative to the distribution of scores for all schools 
in the district. 


Source: Authors’ calculations based on data from the Conditions for Learning Survey. 


Figure 4. English learner students’ performance on state standardized assessments 
from 2011/12 to 2016/17 were below the district average for English language 
arts in all six years of the study and for math in four years of the study 
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Note: The state standardized assessment was the Ohio Achievement Assessment in 2012-14, the Partner- 
ship for Assessment of Readiness for College and Careers in 2015, and the Ohio State Test in 2016 and 
2017. The study team standardized math and English language arts scores relative to the distribution of 
scores across all students in the district in each year and grade level. 


Source: Authors’ analyses based on data from the Cleveland Metropolitan School District. 


in assessments, but the study did not explore whether the changes explain the observed 
year-to-year variations (see appendix B for a more detailed discussion of the state standard- 
ized assessments used during the study period). 


English learner students’ overall English language proficiency levels were generally stable 


There was little change in the proportion of students in grades 3-8 scoring at each level on 
the two state English language proficiency assessments used during the study years. During 
2011/12—2014/15, 3—5 percent of English learner students received an overall score indicat- 
ing full proficiency on the assessment used during that period (figure 5). During 2015/16— 
2016/17, 9-14 percent of students received an overall score indicating full proficiency on 
the assessment used during that period. The increase in the percentage of students achiev- 
ing full proficiency in the last two years coincided with the change in assessment, but the 
study did not explore whether the change in assessment explained the observed increase. 


Figure 5. Overall English language proficiency levels among English learner 
students in grades 3-8 were stable under each state assessment from 2011/12 
to 2016/17 
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Note: The number of overall proficiency levels decreased from five to three between 2014/15 and 2015/16. 
The English language proficiency assessment administered in 2011/12-—2014/15 was the Ohio Test of English 
Language Acquisition, and the assessment administered in 2015/16 and 2016/17 was the Ohio English Lan- 
guage Proficiency Assessment. See appendix B for a more detailed explanation of both assessments. 


Source: Authors’ analyses based on data from the Cleveland Metropolitan School District. 


Special education status and lower prior-year assessment performance were consistently 
associated with lower academic performance and English language proficiency level 


English learner students with special education status had lower math performance, 
English language arts performance, and English language proficiency levels than did 
English learner students without special education status, even after student background 
and school characteristics were accounted for (table 1; see also tables Cll and C12 in 
appendix C). Prior-year math performance, prior-year English language arts performance, 
and prior-year overall English language proficiency level all were positively associated with 
their corresponding spring 2017 student performance measure. This means that students 
with higher outcomes in spring 2016 had higher outcomes in spring 2017 and that students 
with lower outcomes in spring 2016 had lower outcomes in spring 2017. 


These findings are consistent with prior research that also found lower academic perfor 
mance among English learner students with special education status and English learner 
students with lower prior performance, even after differences in student background char- 
acteristics were accounted for (for example, Greenberg Motamedi, 2015; Hass, Huang, 
Tran, & Yu, 2016; Parker, O’Dwyer, & Irwin, 2014; Thompson, 2017). 


English learner students whose native language was Arabic tended to have lower English language 
proficiency levels, while gifted students and female students tended to have higher proficiency levels 


English learner students whose native language was Arabic had lower proficiency levels 
overall and in listening, reading, and writing than did students whose native language 
was Spanish (see table 1; see also table C12 in appendix C). Similarly, students whose 


Table 1. Student characteristics related to English learner math and English 
language arts performance and English language proficiency levels, 2016/17 


Ohio English Language 
Ohio State Test Proficiency Assessment 


English 
language 
Characteristic Math arts Overall Listening Speaking Reading Writing 


Special education status - - = = = = = 


Prior-year assessment score® + + + + + + + 
Native language = Arabic? ns ns - - ns - - 
Native language = African language ns ns ns = ns ns ns 
Native language = Asian language? ns ns ns ns ns ns ns 
Native language = 

European/other language? ns ns + ns + ns ns 
Female ns ns ar ns + ns + 
Gifted status ns ns ap ns ns ns ns 


ns is not significant at the p = .05 level. 
+ and brown shading denotes a statistically significant positive relationship. 
— and grey shading denotes a statistically significant negative relationship. 


a. Associations with prior-year math and English language arts performance do not include grade 3 students 
because they do not take the assessment in grade 2. 


b. Compared with English learner students whose native language is Spanish. 


Source: Authors’ analyses based on the regressions that account for student and school characteristics (see 
tables C12 and C13 in appendix C). 


native language was an African language had lower listening proficiency levels than did 
students whose native language was Spanish. Conversely, students whose native language 
was a European/other language had higher overall and speaking proficiency levels than did 
students whose native language was Spanish. Despite the differences in English language 
proficiency by student native language, there were no clear differences in math or English 
language arts performance by student native language. 


Female students had higher overall, speaking, and writing proficiency levels than did male 
students, and students identified as gifted had higher overall English language proficiency 
levels than did students not identified as gifted (see table 1; see also table C12 in appendix 
C). There were no clear differences in math or English language arts by student gender or 
gifted status. 


The percentage of Spanish-speaking English learner students in a school was associated with math 
performance and with English language overall, listening, and speaking proficiency levels 


Three of the school characteristics studied were associated with math performance (table 2; 
see also table Cll in appendix C). English learner students had higher math performance 
when they attended a school with a higher proportion of Spanish-speaking English learner 
students and larger total student enrollment. Students had lower math performance when 
they attended a school with more English learner students per bilingual paraprofessional. 


Two of the school characteristics studied were associated with English language proficiency 
levels. As with math performance, overall, listening, and speaking proficiency levels were 
higher among students who attended schools with a higher proportion of Spanish-speaking 


Table 2. School characteristics related to English learner math and English 
language arts performance and English language proficiency levels, 2016/17 


Ohio English Language 
Ohio State Test Proficiency Assessment 


English 
language 
Characteristic Math arts Overall Listening Speaking Reading Writing 


Number of English learner 
students per certified English as a 
second language teacher ns ns ns ns = ns ns 


Number of English learner 

students per bilingual 

paraprofessional - ns ns ns ns ns ns 
Percentage of all students who are 

English learner students ns ns = = = ns ns 
Percentage of English learner 


students whose native language is 
Spanish + ns + + + ns ns 


School size = large? ae ns ns ns ns ns ns 


ns is not significant at the p = .05 level. 

+ and brown shading denotes a statistically significant positive relationship. 
— and grey shading denotes a statistically significant negative relationship. 
a. Compared with small or medium schools, as defined in appendix B. 


Source: Authors’ analyses based on the regressions that account for student and school characteristics (see 
tables C12 and C13 in appendix C). 


10 


English learner students and were lower among students who attended schools with higher 
proportions of English learner students overall (see table 2; see also table C12 in appendix 
C). One additional school characteristic studied was associated with speaking proficien- 
cy level: English learner students who attended schools with more students per certified 
English as a second language teacher had lower speaking proficiency levels. 


No school characteristics were clearly associated with English language arts performance 
or English language reading or writing proficiency levels (see table 2; see also tables C11 
and C12 in appendix C). 


School climate domains were positively associated with English language speaking proficiency level 
but not with most other student outcomes 


All four domains of school climate were positively associated with English language 
speaking proficiency level (table 3; see also table C14 in appendix C). In addition, English 
learner students who attended schools with higher student ratings of academic rigor and 
supportive learning environment had higher English language arts performance. Students 
who attended schools with higher safe and respectful climate ratings had higher listening 
proficiency levels. There were no other clear associations between the school climate mea- 


sures and the other student outcomes. 


Implications of the study findings 


The study findings provide direction for further investigation into actionable steps for 
CMSD to support English learner students’ academic performance and English language 
proficiency. While the results speak to these relationships in just one district, educators 
and policymakers from other geographic areas may consider similar studies to determine 
whether the relationships are similar in their own contexts. 


The study findings suggest the need for additional examination of the role of school climate 
in student learning. The associations between English language speaking proficiency and 
student perceptions of school climate could be investigated further to better understand 


Table 3. Conditions for Learning Survey measures related to math and English 
language arts performance and English language proficiency levels of English 
learner students, 2016/17 


Ohio English Language 


Ohio State Test Proficiency Assessment 

English 

language 
Domain WEL) arts Overall Listening Speaking Reading Writing 
Academic rigor ns sb ns ns 3h ns ns 
Safe and respectful climate ns ns ns + + ns ns 
Peer social-emotional learning ns ns ns ns ae ns ns 
Supportive learning environment ns + ns ns + ns ns 


ns is not significant at the p = .05 level. 
+ denotes a statistically significant positive relationship. 


Source: Authors’ analyses based on the regressions that account for student and school characteristics (see 
tables C14 and C15 in appendix C). 
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whether the relationships are causal. For example, it could be useful for the district to 
examine whether a positive school climate contributes to the ability of English learner 
students to improve their speaking skills. Schools may also wish to consider whether there 
are approaches to building a positive school climate that are particularly well suited for 
supporting English learner students. Further research could focus on working with the dis- 
trict to evaluate school climate interventions that appear to be useful for improving the 
performance of English learner students. 


Relatedly, it would be useful to understand the extent to which the newcomer academy’s 
particularly high ratings for school climate reflect strong practices for building a positive 
school climate that could be replicated in other schools. The district also may wish to 
understand whether newcomer academy student supports help explain school climate per- 
ceptions and student performance. The district also might benefit from studying English 
learner school climate ratings over time, as students leave the newcomer academy and 
acclimate to their new contexts. 


Further, these findings contribute to the body of research suggesting that school climate 
is a promising area of focus for districts and schools. While research on school climate as 
it pertains specifically to English learner students is limited (O’Conner, De Feyter, Carr, 
Luo, & Romm, 2017), the findings of this study are similar to those of previous studies 
in other locations based on general student populations that have found an association 
between school climate perceptions and student learning and performance (for example, 
Durlak, Weissberg, Dymnicki, Taylor, & Schellinger, 2011; Hopson, Schiller, & Lawson, 
2014). In CMSD, school climate has been a long-standing district priority (Osher, Poirier, 
Jarjoura, Haight, & Mitchell, 2014). The current study provides a step toward understand- 
ing the association between school climate and English learner student performance and 
raises new questions, such as why speaking proficiency was most consistently associated 
with Conditions for Learning Survey measures. Future research could use a more rigorous 
approach to understand whether districts and schools can improve school climate as a 
pathway to improving performance for English learner students and all students. 


Finally, school staffing characteristics demonstrated some associations with student out- 
comes, but the findings were not consistent, so these potential relationships require further 
study. For example, the district may benefit from examining how changes in the number 
of English learner students per certified English as a second language teacher or bilingual 
paraprofessional in a school are related to student outcomes. This study found that a 
higher number of English learner students per bilingual paraprofessional was associated 
with lower math performance. This finding is similar to that in other research, which 
has found positive associations between students having access to teachers with bilingual 
certification and student performance (Loeb, Soland, & Fox, 2014; Ruiz de Castilla, 2018). 
Additionally, student English language speaking proficiency levels (although not other 
student outcomes) were higher in schools with fewer students per English as a second lan- 
guage teacher and were lower in schools with no certified English as a second language 
teachers. This contrasts somewhat with previous research in the Houston Independent 
School District, which found no clear associations between teacher certification in English 
as a second language and English learner student performance (Ruiz de Castilla, 2018). 
Taken together, these results suggest that further research is needed into the number and 
type of staff that schools employ to support English learner students and their outcomes. 
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Limitations of the study 


This study has five key limitations. 


First, the study team would have liked to examine how student and school characteristics 
were related to student performance over time, but assessment data over multiple years 
were not available for enough students to allow for a longitudinal approach. In addition, 
English learner student performance on statewide assessments relative to the district 
average shifted as the assessments changed, further limiting the ability to conduct a longi- 
tudinal study. 


Second, the study would be improved with an understanding of how long a student had 
been in the district and, further, how long a student had been in the United States. For 
example, it is possible that Spanish-speaking students were more likely to have been in 
the district longer than students from other language groups. But this information was not 
available from district records. 


Third, while the analysis includes school-level information such as the ratio of students 
to specialist staff, CMSD does not track studentlevel records about the specific English 
learner services that students receive. This limited the study team’s ability to understand 
variation in student services in a school or school type. 


Fourth, the study focused on English learner students in a single school district, limiting 
the generalizability of the study. Although CMSD is likely similar to other urban districts 
in Ohio and the REL Midwest Region more broadly,’ the results may not generalize to 
districts that have larger populations of English learner students than CMSD. This study 
also focused on students in grades 3—8; therefore, the results also may not be relevant for 
English learner students in other grade levels. 


Fifth, the correlational design for this study cannot determine whether student and 
school characteristics play a causal role in English learner student outcomes. Findings are 
described as associations and suggest areas for further investigation, but they do not sup- 
plant the need for research with a different design to assess actual impacts. 
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Appendix A. Review of the literature 


US. civil rights law, following Lau v. Nichols (1974), mandates that education programs 
meet both the language and content needs of English learner students. Meeting these 
needs is often a long-term process. English learner students exhibit lower content profi- 
ciency in math and reading at school entry and throughout elementary and high school 
(Flores, Batalova, & Fix, 2012; Fry, 2007; Reardon & Galindo, 2009; Slama, 2014). 


Since 2003 the National Assessment of Educational Progress has documented significant 
national performance gaps between English learner students and non-English learner stu- 
dents in both math and reading at all age levels tested, with no indication of improvement 
in English learner student performance in either content area at any grade level through 
2013 (U.S. Department of Education, National Center for Education Statistics, n.d.). 


Compared with non-English learner students, English learner students consistently under- 
perform in math and English language arts (Cook, Boals, & Lundberg, 2011; Fry, 2007; 
Menken, 2008; Reardon & Galindo, 2009; Ruiz-de-Velasco, Fix, & Clewell, 2000; Valencia 
& Villarreal, 2005) and often are considered at risk because they must learn academic 
content concurrently to gaining language proficiency (Harper, de Jong, & Platt, 2008; 
Short & Fitzsimmons, 2007). Research suggests that it can take up to 10 years for newly 
arriving English learner students to develop English proficiency (Batalova, Fix, & Murray, 
2007; Cook, Boals, Wilmes, & Santos, 2008; Hakuta, Butler, & Witt, 2000; Linquanti & 
George, 2007). 


English learner students are a diverse group, and thus it is important that instructional 
approaches take into account their diverse needs. Olsen (2010) found that 59 percent of 
English learner students in California secondary schools had been in a U.S. school for 
more than six years without reaching a sufficient level of English proficiency to be reclas- 
sified as fluent. The study also found that few school districts had programs designed for 
long-term English learner students. Other English learner students are newly arrived stu- 
dents (immigrants or refugees) who may have limited English proficiency and special needs 
such as health issues as they face challenges at the higher grade levels upon arrival (Short 
& Boyson, 2012). 


Mainstream teachers can lack knowledge related to educating English learner students. 
More than 30 states have no training requirements for teachers to work with English 
learner students, and none of the Regional Educational Laboratory Midwest states has 
such a requirement (Education Commission of the States, 2014).4 Specific instructional 
knowledge and skills are required to effectively teach English learner students (Master, 
Loeb, Whitney, & Wyckoff, 2016), yet general education teachers often lack these skills 
(Lucas & Grinberg, 2008) and report feeling underprepared to effectively teach English 
learner students (Cho & McDonnough, 2009). Darling-Hammond, Wei, Andree, Richard- 
son, & Orphanos (2009, p. 6) indicated that “more than two-thirds of teachers nationally 
had not had even one day of training in supporting the learning of special education or 
[English learner] students during the previous three years, and only one-third agreed that 
they had been given the support they needed to teach students with special needs.” 


Districts and schools may seek to understand how student and school characteristics influ- 
ence English learner student success, and research has provided some clues. Numerous 
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studies have documented associations between English learner student performance and 
student characteristics, including immigrant status, special education designation, race/ 
ethnicity, and poverty status (for example, Glick & Hohmann-Marriott, 2007; Leventhal, 
Xue, & Brooks-Gunn, 2006; Mulligan, Halle, & Kinukawa, 2012). For instance, higher 
socioeconomic status has been associated with shorter time to English language proficien- 
cy across multiple districts (Hakuta et al., 2000). Other research has found that special 
education status was significantly associated with lower English proficiency among English 
language learner students (Parker et al., 2014). 


Also, it is particularly important to understand how school characteristics are associat- 
ed with English learner student outcomes because districts and schools have control over 
some of these characteristics. School characteristics and the kinds of first language support 
that English learner students receive have been found to have significant associations with 
Hispanic English learner student and native English-speaking student performance gaps 
(Han, 2012) as well as with English learner students’ short- and long-term skill develop- 
ment (Collier, 1992; Umansky & Reardon, 2014). 


Several studies have examined the relationship between teacher certification in English 
as a second language and English learner student performance. In a study based in the 
Houston Independent School District, student improvement in math and reading perfor 
mance and English language proficiency was not associated with having a teacher with 
English as a second language certification (Ruiz de Castilla, 2018). Similarly, Parker et al. 
(2014) found no relationship between English language proficiency and the percentage of 
English learner students at a school taught by certified English as a second language teach- 
ers. However, Lopez, Scanlan, and Gundrum (2013) analyzed reading performance among 
Hispanic students on the National Assessment of Educational Progress and found higher 
performance among students in states with more stringent requirements for earning certi- 
fication in educating English learner students. 


Research also has found generally positive relationships between teacher bilingual certifi- 
cation and student performance (Loeb et al., 2014; Ruiz de Castilla, 2018), although there 
is a lack of research examining the relationship between bilingual paraprofessionals and 
student outcomes. The role of bilingual paraprofessionals in English learner student out- 
comes merits further research, given the potential of paraprofessionals to address common 
shortages of bilingual certified teachers and to create career pathways for a more diverse 
teaching force (Amos, 2013). 
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Appendix B. Study methodology 


This study addressed questions posed by the Cleveland Partnership for English Learner 
Success, a partnership between researchers from the Regional Educational Laboratory 
Midwest and the Cleveland Metropolitan School District (CMSD). This appendix pro- 
vides further details on the study’s data sources, sample, missing data, sensitivity analysis, 
measures, variables, and analysis methods. 


Data sources 


CMSD maintains student administrative records on a yearly basis to supplement the Condi- 
tions for Learning Survey data collected by the American Institutes for Research (AIR). AIR 
and CMSD have an ongoing data-sharing agreement that allows AIR to use administrative 
data as well as student and school-level Conditions for Learning Survey data to conduct studies 
with permission from the district on a case-by-case basis. The study team obtained informa- 
tion from CMSD pertaining to staff full-time equivalency in the district over the six-year study 
period. School-level data, including student enrollment statistics, were obtained from publicly 
available, school-level data files on the website of the Ohio Department of Education (2018). 


Sample 


The sample for the current study included all English learner students in grades 3-8 from 2011/12 
to 2016/17 and all schools enrolling those students each year (table B1). Any school enroll 
ing one or more English learner students in grades 3—8 from 2011/12 to 2016/17 was included. 


CMSD tests all English learner students each year. Thus, the study team identified stu- 
dents as English learner students by whether they had taken an English language prof- 
ciency assessment in a given year. Any student who had a score on either the Ohio Test 
of English Language Acquisition (in 2011/12—2014/15) or the Ohio English Language Pro- 
ficiency Assessment (in 2015/16 and 2016/17) was considered an English learner student. 


Measures 


To answer research question 1 about the characteristics of English learner students and the 
schools they attended, the study team used the variables shown in table B2. 


Table B1. Number of English learner students and schools enrolling English learner 
students in the Cleveland Metropolitan School District, 2011/12-2016/17 


Number of Number of schools enrolling 
English learner students English learner students 
AYod Cele) AY 1 in grades 3-8 in grades 3-8 
2011/12 1,266 42 
2012/13 1,237 42 
2013/14 1,215 41 
2014/15 1,310 44 
2015/16 1,585 47 
2016/17 1,542 41 


Note: English learner students were identified by whether they took an English language proficiency assess- 
ment in a given year. 


Source: Authors’ calculations based on data from the Cleveland Metropolitan School District. 
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Table B2. Student- and school-level variables used in analyses 


Variable 


Student characteristics 


English language proficiency 
level 


Description 


Proficiency level in listening to English and speaking, reading, and writing 
in English as assessed by the Ohio Test of English Language Acquisition 
(2011/12-2014/15) or the Ohio English Learner Proficiency Assessment 
(2015/16 and 2016/17). 


Native language 


Coded as Spanish; Arabic; African language (Bassa, Dinka, Ewe, Igbo, 
Karen, Kinyamulenge, Kirundi, Krahn, Maay Maay, Somali, Swahili, Tigrinya, 
and Yoruba); Asian language (Burmese, Cambodian, Cantonese, Filipino, 
Hindi, Indonesian, Mandarin, Nepali, Punjabi, Tagalog, Tamil, Thai, Urdu, 
and Vietnamese); or European/other language (Albanian, Bosnian, Croatian, 
English/Creole, French, Romanian, Russian, Turkish, and Ukrainian). 


Race/ethnicity 


Gender 


Coded as Asian, Black, Hispanic, White, or other 


Coded as female or male 


Special education status 


Whether a student received special education services for a learning 
disability 


Gifted status 


Whether a student received gifted and talented services 


Grade level 


Grade level at the end of the year 


Math and English language 
arts performance 


School characteristics 
School type 


Score on state standardized assessments, standardized using annual 
districtwide grade-level means and standard deviations 


Coded as bilingual school, nonmultilingual school, or the district’s newcomer 
academy 


Number and percentage of 
English learner students 


Raw number and proportion of students in a school who are identified as 
English learner students 


Number of languages spoken 
among English learner 
students 


Number of unique native languages spoken among English learner students 
in a school 


Percentage of English learner 
students whose native 
language is Spanish 


Percentage of English learner students in a school who are also identified as 
Spanish speakers 


Percentage of English learner 
students whose native 
language is Arabic 


Percentage of English learner students in a school who are also identified as 
Arabic speakers 


School specialist staffing 


Number of English learner students per certified English as a second 
language teacher and number of English learner students per bilingual 
paraprofessional in a school 


School climate measures 


Based on student responses to the Conditions for Learning Survey, which 
consists of four domains: academic rigor, safe and respectful climate, peer 
social-emotional learning, and supportive learning environment 


Percentage of English learner 
students at each English 
language proficiency level 


Percentage of English learner students in a school who tested at each 
English language proficiency level. 


Percentage of all students 
who met proficiency 
standards in math and 
English language arts 


Percentage of all students in a school who scored proficient or higher on the 
state standardized assessment 


School size 


Coded as small, medium, or large based on the annual distribution of 
enrollment across the district’s K-8 schools 


Other school-level 
descriptors 


Percentage of all students in a school in each racial/ethnic category, 
percentage of all students in a school who have special education status, 
percentage of all students in a school who have gifted status 


Source: Authors’ compilation. 


Student-level variables 


English language proficiency level. All English learner students, when they are onboarded 
into the district each fall, as well as each spring until they are reclassified (or no longer 
in need of special language assistance or instruction), are assessed by the district for their 
English language proficiency in listening, speaking, reading, and writing using a state stan- 
dardized assessment. In addition to being scored in each of these areas, students are given 
an overall, composite score that determines their English language proficiency level. The 
assessment has forms that are used for specific grade-level bands that align with the Ohio 
English Language Proficiency Standards. Accordingly, there are forms for grades K, 1, 2—3, 
4—5, 6-8, and 9-12. The same proficiency levels are used in each grade-level band form of 
the assessment. 


The English language proficiency assessment changed during the study period. In 2011/12— 
2014/15 students took the Ohio Test of English Language Acquisition, whose overall 
and subscale scores map to five performance levels: 1—prefunctional, 2—beginning, 
3— intermediate, 4—advanced, and 5—full English proficiency skills. In 2015/16 and 
2016/17 students took the Ohio English Learner Proficiency Assessment, whose sub- 
scale scores map to five performance levels: 1—beginning, 2—early intermediate, 
3— intermediate, 4—early advanced, and 5—advanced skills. Based on the performance 
level for the four subscales, students are then assigned an overall proficiency level of 
l—emerging, 2—progressing, or 3—proficient. The performance level cutpoints are as 
follows (Ohio Department of Education, 2017): 
e Emerging means that a student scored any combination of level 1s and 2s on the 
four subscales. 
e Progressing means that a student scored a combination of levels that did not allow 
the student to be considered proficient or emerging. 
e Proficient means that a student scored any combination of level 4s and 5s on the 
four subscales. 


The study team used the grade-level band English language proficiency-level designations. 


Native language. CMSD keeps detailed records of the language reported by a student’s 
parents or guardian as the student’s primary native language. The records include more 
than 70 distinct language codes. For ease of reporting, the study team grouped these lan- 
guages as follows: 
e Spanish. 
e Arabic. 
e African languages: Bassa, Dinka, Ewe, Igbo, Karen, Kinyamulenge, Kirundi, 
Krahn, Maay Maay, Somali, Swahili, Tigrinya, and Yoruba. 
e Asian languages: Burmese, Cambodian, Cantonese, Filipino, Hindi, Indonesian, 
Mandarin, Nepali, Punjabi, Tagalog, Tamil, Thai, Urdu, and Vietnamese. 
e European/other languages: Albanian, Bosnian, Croatian, English/Creole, French, 
Romanian, Russian, Turkish, and Ukrainian. 


Twenty-seven other languages were spoken by CMSD students who were not in the analyt 
ical sample. Those languages were not categorized here. 


Race/ethnicity. CMSD data also provide information on student race/ethnicity. The study 
team grouped students into five categories: Asian, Black (includes African American), 
Hispanic (includes Latino), White, and other (includes Hawaiian or other Pacific Islander, 
American Indian/Alaska Native, and two or more races/ethnicities). 


Gender. The study team constructed student gender as a dichotomous indicator. Female 
students were designated as 1, and male students were designated as 0. 


Special education status. Students who qualify for special education services have their 
needs designated under nine special education codes.* The study team constructed special 
education status as a dichotomous indicator, whether students qualify for special education 
services or not. 


Gifted status. Students identified as gifted and talented are flagged in district administra- 
tive records. The study team constructed gifted status as a dichotomous indicator, whether 
students were designated as gifted or not. 


Grade level. The study team identified students’ grade level in a given year by the grade in 
which they were reported to be at the end of the school year. 


Math and English language arts performance. All students in grades 3-8 in CMSD, includ- 
ing English learner students, are required to take the state achievement assessment each 
spring. The assessment changed during the study period: the Ohio Achievement Assess- 
ment was administered in 2012-14, the Partnership for Assessment of Readiness for 
College and Careers was administered in 2015, and the Ohio State Test was administered 
in 2016 and 2017. To facilitate examination of patterns over time and across assessments, 
the study team used annual districtwide, grade-level math and English language arts means 
and standard deviations to standardize student scores, which allowed examination of how 
English learner students performed relative to all students in the district each year. 


School-level variables 


School type. The district offers three schools that differ in the type of language program 
offered: bilingual schools (seven K—8 schools), nonmultilingual schools (67 K—8 schools), 
and a newcomer academy (one K-12 school). 


Number and percentage of English learner students. The study team constructed variables to 
capture the proportion of English learner students in the district as well as the proportion 
of English learner students, the proportion of Spanish-speaking English learner students, 
and the proportion of Arabic-speaking English learner students in each school included in 
the analysis for each year of the study. 


Number of languages spoken among English learner students. The study team constructed 
this variable to capture the number of different native languages spoken among English 
learner students at a school. The variable does not include languages spoken by students 
who are not classified as English learner students in a specific year. At the school level the 
study team considered the total number of languages spoken in a school rather than the 
language groups used at the student level. For example, at the school level the study team 
counted Swahili and Yoruba separately rather than as African languages. 
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Percentage of English learner students whose native language is Spanish. The study team con- 
structed this variable to capture the proportion of English learner students at a school that 
were identified as Spanish speakers. 


Percentage of English learner students whose native language is Arabic. The study team con- 
structed this variable to capture the proportion of English learner students at a school that 
were identified as Arabic speakers. 


School specialist staffing. The study team constructed the number of English learner stu- 
dents per certified English as a second language teacher and the number of English learner 
students per bilingual paraprofessional for each school based on staffing data from Multi- 
lingual Multicultural Education Office administrative records for all six years. 


School climate measures. The study team analyzed student responses to the Conditions for 
Learning Survey (Osher et al., 2008) for each school. The Conditions for Learning Survey 
is a psychometrically validated instrument for measuring student connections and condi- 
tions for learning at the elementary school (grades 2—4), middle school (grades 5-8), and 
high school (grades 9-12) levels. The survey is administered in the fall, winter, and spring 
of each year to all students in grades 2-12 who are not identified as needing alternative 
accommodations for test taking and assessment (due to, for example, severe cognitive dis- 
ability). The survey is administered in English and Spanish, and items are read aloud to 
students in grades 2—4, as well as in a student’s native language in any grade when neces- 
sary. Response rates for elementary and middle school grades are consistently at or above 
90 percent. Since the survey was developed and first administered in Chicago in 2007, it 
has been administered to or adapted for public school students in several school districts 
around the country, including CMSD. The survey is designed to assess four core constructs 
that measure the conditions for learning, or climate, within a school: 


1. Academic rigor asks students about the level of challenge and the school’s expecta- 
tions for student academic success (17 items, Cronbach’s « = 0.85). 


2. Safe and respectful climate captures student perceptions of how emotionally and phys- 
ically safe they feel at school (15 items, a = 0.89). 


3. Peer social-emotional learning measures the extent to which students feel their peers 
are able to handle social-emotional challenges (11 items, # = 0.84). 


4. Supportive learning environment measures the extent to which students feel connect- 
ed and attached to adults in the school and believe that their teachers care about them 
and treat them respectfully (14 items, a = 0.82). 


The Conditions for Learning Survey asked students their perceptions of each of the afore- 
mentioned domains in their school environment, and their responses are aggregated to 
produce school-level measures of school climate for each school. Survey data are available 
for the five most recent years of the study (2012/13—2016/17). 


The study team created school-level measures for all four domains by averaging the ele- 
mentary and middle grade scores for each school. The study team then created a composite 
measure of school climate by summing each school’s averaged domains. All four domains, 
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in addition to the composite school climate measure, were standardized to the districtwide 
distribution in each school year. 


Percentage of English learner students at each English language proficiency level. The study 
team constructed school-level measures related to the percentage of students who tested 
at each proficiency level on the state’s two assessments of English learner proficiency—the 
Ohio Test of English Language Acquisition in 2011/12—2014/15 and the Ohio English Lan- 
guage Proficiency Assessment in 2015/16 and 2016/17. 


Percentage of all students who met proficiency standards in math and English language arts. 
The state classifies student performance on standardized assessments in math and English 
language arts into five categories: limited, basic, proficient, accelerated, and advanced. The 
study team collapsed these categories to construct a dichotomous variable, proficient (pro- 
ficient, accelerated, or advanced) or not proficient (limited or basic) in each subject. Then, 
a schoolwide average was created based on the proportion of students in the school who 
were proficient in each subject. 


School size. The study team categorized schools into three groups based on the distribution 
of total student enrollment for all K—8 schools in the district, determined within each 
year. The team characterized schools with enrollment under the 25th percentile as small, 
schools with enrollment in the 25th—75th percentiles as medium, and schools with enroll- 
ment over the 75th percentile in enrollment as large. In 2016/17 the 25th percentile was 
315 students, and the 75th percentile was 475 students. 


Other school-level descriptors. To further describe the school context, other school-level 
data were examined, including percentage of all students in each racial/ethnic category, 
percentage of all students with special education status, and percentage of all students with 
gifted status. These variables were aggregated from student-level administrative records. 


Missing data 


Rates of missing data in the sample for the descriptive analysis were generally low 
(4.5 percent or less; table B3). School-level data were missing from the samples for several 
reasons. Some data were not available at all for a specific year (for example, information on 
certified English as a second language teachers in 2011/12—2015/16 and mean scores on the 
Conditions for Learning Survey in 2011/12). 


Missing data in the sample for the regression analysis was specific to prior-year (spring 
2016) academic performance, which was due primarily to students being new to the dis- 
trict or being in grade 3 and thus not having prior-year math and English language arts 
assessment information. The study team replaced missing values with zeros and included 
an indicator variable in all analyses to signal whether prior-year performance was missing. 
Rates of missing data for student performance on the spring 2017 Ohio State Test math 
or English language arts assessments are displayed in table B4. Rates of missing data for 
English language proficiency level on the spring 2017 Ohio English Language Proficiency 
Assessment are displayed in table B5. 
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Table B3. Rates of missing student- and school-level data in the analytic samples for descriptive analyses, 2011/12-2016/17 


Characteristic 
Student characteristics 
English language proficiency level 


2011/12 


of valid 
cases 


Percent 
missing 


2012/13 


of valid 
cases 


Percent 
missing 


2013/14 


of valid 
cases 


Percent 
missing 


2014/15 


of valid 
cases 


Percent 
missing 


2015/16 


of valid 
cases 


Percent 
missing 


2016/17 


of valid 
cases 


Percent 
missing 


Overall 1,223 3.4 1,205 2.6 1,199 1.3 1,289 1.6 1,582 0.2 1,537 0.3 
Listening 1,258 0.6 1,226 0.9 1,209 0.5 1,299 0.8 1,581 0.3 1,527 1.0 
Speaking 1,241 2.0 1,222 1.2 1,209 0.5 1,299 0.8 1,562 1.5 1,516 1.7 
Reading 1,257 0.7 1,230 0.6 1,211 0.3 1,306 0.3 1,575 0.6 1,529 0.8 
Writing 1,251 1.2 1,223 1.4 1,206 0.7 1,298 0.9 1,569 1.0 1,527 1.0 
Native language 1,266 0.0 1,237 0.0 1,215 0.0 1,310 0.0 1,585 0.0 1,542 0.0 
Race/ethnicity 1,266 0.0 1,237 0.0 1,215 0.0 1,310 0.0 1,585 0.0 1,542 0.0 
Gender 1,266 0.0 1,237 0.0 1,215 0.0 1,310 0.0 1,585 0.0 1,542 0.0 
Special education status 1,266 0.0 1,237 0.0 1,215 0.0 1,310 0.0 1,585 0.0 1,542 0.0 
Gifted status 1,266 0.0 1,237 0.0 1,215 0.0 1,310 0.0 1,585 0.0 1,542 0.0 
Math performance 1,210 4.4 1,183 4.4 1,175 3.3 1,251 4.5 1,549 2.3 1,475 4.4 
English language arts performance 1,210 4.4 1,183 4.4 1,174 3.4 1,256 4.1 1,548 2.3 1,481 4.0 
School type 1,260 0.5 1,230 0.5 1,211 0.3 1,308 0.2 1,584 0.1 1,542 0.0 
School characteristics 
Percentage of students who are English 
learner students 42 0.0 42 0.0 41 0.0 44 0.0 47 0.0 41 0.0 
Percentage of English learner students 
whose native language is Spanish 42 0.0 42 0.0 41 0.0 44 0.0 47 0.0 41 0.0 
Percentage of English learner students 
whose native language is Arabic 42 0.0 42 0.0 41 0.0 44 0.0 47 0.0 41 0.0 
Number of English learner students per 
certified English as a second language 
teacher ) 100.0 fe) 100.0 0) 100.0 0) 100.0 0) 100.0 41 0.0 
Number of English learner students per 
bilingual paraprofessional 42 0.0 42 0.0 41 0.0 44 0.0 47 0.0 41 0.0 
Conditions for Learning Survey scores 0 100.0 42 0.0 41 0.0 44 0.0 47 0.0 41 0.0 
Percentage of English learner students at 
each English language proficiency level 42 0.0 42 0.0 41 0.0 44 0.0 47 0.0 41 0.0 
Percentage of all students who met math 
proficiency standards fe) 100.0 41 2.4 40 2.4 44 0.0 46 2.1 41 0.0 


(continued) 
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Table B3. Rates of missing student- and school-level data in the analytic samples for descriptive analyses, 2011/12-2016/17 (continued) 


2011/12 2012/13 2013/14 2014/15 2015/16 2016/17 


of valid Percent _ of valid Percent _ of valid Percent of valid Percent COMTELIC Percent OOMELIC Percent 
Characteristic cases missing cases missing cases missing cases missing cases missing cases missing 


Percentage of all students who met 
English language arts proficiency 


standards 42 0.0 42 0.0 41 0.0 44 0.0 47 0.0 41 0.0 
School size 42 0.0 42 0.0 41 0.0 44 0.0 47 0.0 41 0.0 
Percentage of all students in each racial/ 

ethnic category 42 0.0 42 0.0 41 0.0 44 0.0 47 0.0 41 0.0 
Percentage of all students with special 

education status 42 0.0 42 0.0 41 0.0 44 0.0 47 0.0 41 0.0 
Percentage of all students with gifted 

status 42 0.0 42 0.0 41 0.0 44 0.0 47 0.0 41 0.0 


Source: Authors’ analyses. 


Table B4. Rates of missing student- and school-level data in the analytic samples 
for student performance regression analyses, 2016/17 


Math English language arts 


Number of Percent Number of Percent 
Characteristic valid cases missing valid cases missing 


Student characteristics 
Prior-year overall English language 


proficiency level 1,126 23.7 1,128 23.8 
Native language 1,475 0.0 1,481 0.0 
Gender 1,475 0.0 1,481 0.0 
Special education status 1,475 0.0 1,481 0.0 
Gifted status 1,475 0.0 1,481 0.0 
Prior-year math performance 937 36.5 na na 


Prior-year English language arts 

performance na na 938 36.7 
School characteristics 

Percentage of all students who are 

English learner students 1,475 0.0 1,481 0.0 
Percentage of English learner 

students whose native language is 

Spanish 1,475 0.0 1,481 0.0 
Number of English learner students 

per certified English as a second 


language teacher 1,475 0.0 1,481 0.0 
Number of English learner students 

per bilingual paraprofessional 1,475 0.0 1,481 0.0 
Measure of school climate 

(standardized) 1,475 0.0 1,481 0.0 
Percentage of all students who met 

math proficiency standards 1,475 0.0 1,481 0.0 


Percentage of all students who met 
English language arts proficiency 


standards 1,475 0.0 1,481 0.0 
School size 1,475 0.0 1,481 0.0 
Percentage of all students with 

special education status 1,475 0.0 1,481 0.0 


na is not applicable because the characteristic was not included in the regression analysis for the indicated 
student outcome. 


Source: Authors’ analyses. 
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Table B5. Rates of missing student- and school-level data in the analytic samples for English language proficiency regression analyses, 
2016/17 


Listening Speaking Reading Writing 


Number of Percent Number of Percent Number of Percent Number of Percent Number of Percent 
Characteristic valid cases missing valid cases missing valid cases missing valid cases missing valid cases missing 


Student characteristics 
Prior-year overall English language 


proficiency 1,159 24.6 1,154 24.4 1,147 24.3 1,154 24.5 1,154 24.4 
Prior-year English language proficiency 

listening level 1,159 24.6 1,154 24.4 1,147 24.3 1,154 24.5 1,154 24.4 
Prior-year English language proficiency 

speaking level 1,143 25.6 1,139 25.4 1,133 25.3 1,140 25.4 1,140 25.3 
Prior-year English language proficiency 

reading level 1,154 24.9 1,150 24.7 1,143 24.6 1,151 24.7 1,151 24.6 
Prior-year English language proficiency 

writing level 1,152 25.1 1,148 24.8 1,142 24.7 1,149 24.9 1,148 24.8 
Native language 1,537 0.0 1,527 0.0 1,516 0.0 1,529 0.0 1,527 0.0 
Gender 1,537 0.0 1,527 0.0 1,516 0.0 1,529 0.0 1,527 0.0 
Special education status 1,537 0.0 1,527 0.0 1,516 0.0 1,529 0.0 1,527 0.0 
Gifted status 1,537 0.0 1,527 0.0 1,516 0.0 1,529 0.0 1,527 0.0 
Prior-year math performance 958 37.5 1,154 24.4 1,147 24.3 956 37.5 956 37.4 


School characteristics 
Percentage of all students who are 


English learner students 1,537 0.0 1,527 0.0 1,516 0.0 1,529 0.0 1,527 0.0 
Percentage of English learner students 
whose native language is Spanish 1,537 0.0 1,527 0.0 1,516 0.0 1,529 0.0 1,527 0.0 


Number of English learner students per 
certified English as a second language 


teacher 1,537 0.0 1,527 0.0 1,516 0.0 1,529 0.0 1,527 0.0 
Number of English learner students per 

bilingual paraprofessional 1,537 0.0 1,527 0.0 1,516 0.0 1,529 0.0 1,527 0.0 
Measure of school climate (standardized) 1,537 0.0 1,527 0.0 1,516 0.0 1,529 0.0 1,527 0.0 
Percentage of all students who met math 

proficiency standards 1,537 0.0 1,527 0.0 1,516 0.0 1,529 0.0 1,527 0.0 


Percentage of all students who met 
English language arts proficiency 


standards 1,537 0.0 1,527 0.0 1,516 0.0 1,529 0.0 1,527 0.0 
School size 1,537 0.0 1,527 0.0 1,516 0.0 1,529 0.0 1,527 0.0 
Percentage of all students with special 

education status 1,537 0.0 1,527 0.0 1,516 0.0 1,529 0.0 1,527 0.0 


Source: Authors’ analyses. 


Sensitivity analyses 


The study team conducted several analyses to examine whether the findings were mean- 
ingfully affected if the variables and samples in the regression model were changed. This 
included re-running analyses on distinct samples based on prior-year English learner profi- 
ciency level, running analyses in which school staffing variables were replaced with indi- 
cators for school type, re-running analyses and excluding students who lacked prior-year 
assessment data (instead of using mean imputation), and testing the analyses using 2016 
performance as the outcome. In all cases the main findings were consistent, or the sample 
sizes in the test analyses were too small to draw definitive conclusions. 


Analysis methods 
The analysis employed a combination of descriptive and correlational methods. 


Research question 1 used means and percentages to describe the characteristics of the 
English learner student population and the schools they attended by year, using the vari- 
ables described in the measures section above. For the student-level description, each year- 
based sample was defined by selecting all students who were identified as English learner 
students and enrolled in grades 3-8 for each year. For the school-level description, each 
yeat-based sample included all the schools that the sample of English learner students 
attended in that year. 


The analysis for research question 2 also used means and percentages to describe the 
patterns in academic outcomes among English learner students in grades 3—8 in the six 
cross-sectional samples. State math and English language arts assessment scores were stan- 
dardized using the annual districtwide, grade-level means and standard deviations. 


To answer research question 3 on the extent to which student and school characteristics 
are associated with English learner student outcomes, the study team developed a series 
of regression models to estimate the association between student performance in 2016/17 
and student and school characteristics. The regression analyses focused on the most recent 
school year in the sample (2016/17), and the sample was restricted to students with outcome 
data (math and English language arts performance and English language proficiency level) 
for 2016/17. Missing values of covariates (see tables B3 and B4) were replaced with a value 
of zero, and a dichotomous flag was created as a missing indicator for each covariate with 
missing data in the analysis. 


The core model was specified as follows: 


Yay a Wee 
where Y,, is the outcome for student i in school j, X, is a vector that captures student 
characteristics, W, is a vector that captures school characteristics, risa school-level ran- 
dom-error term, and €,, is an individual random-error term. All models were estimated as 
two-level, hierarchical linear models to account for the nesting of students within schools. 
State math and English language arts assessment scores were standardized using annual 
districtwide, grade-level means and standard deviations; thus, coefficients can be interpret- 
ed in standard deviation units. 
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The study team explored using ordinal logistic estimation for English language proficiency 
assessment outcomes given that these had discrete, ordered integer values; the patterns 
of significance and direction of the estimated relationships were the same as when linear 
models were used; thus, the results from the linear models are presented for ease of inter- 
pretation. Sensitivity analyses were conducted to examine the results using complete cases 
only, and no substantive differences were found. 
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Appendix C. Additional results 


This appendix details full results of the findings from the three research questions 
addressed in the study. 


Table C1. Summary characteristics of English learner students in grades 3-8 in the 
Cleveland Metropolitan School District, 2011/12-2016/17 


Characteristic 2011/12 2012/13 2013/14 2014/15 2015/16 2016/17 


Number of English learner students 1,266 1,237 1,215 1,310 1,585 1,542 
Percentage of all students who are 
English learner students 6 7 7 8 9 9 
Percentage of students by native language 
Spanish 82 80 79 79 76 71 
Arabic 4 5 6 6 8 9 
African language® 5 6 6 5 7 10 
Asian language” 6 6 7 8 7 7 
European/other language° 2 1 1 1 1 2 
Other or unknown 1 1 1 1 1 1 
Percentage of students by race/ethnicity 
Asian 6 6 7 8 7 7 
Black 3 4 4 5 8 11 
Hispanic 82 80 79 79 76 71 
White 5 5 6 6 8 10 
Other 4 4 3 2 1 1 
Percentage of students who are 
female 47 48 47 46 46 46 
Percentage of students with special 
education status 16 15 16 18 18 17 
Percentage of students with gifted 
status 2 1 1 2 3 3 
Percentage of students in each school type 
Bilingual school 70 69 64 59 56 54 
Nonmultilingual school 19 20 21 25 21 22 
Newcomer academy 11 11 15 17 23 23 


a. Includes Bassa, Dinka, Ewe, Igbo, Karen, Kinyamulenge, Kirundi, Krahn, Maay Maay, Somali, Swahili, 
Tigrinya, and Yoruba. 


b. Includes Burmese, Cambodian, Cantonese, Filipino, Hindi, Indonesian, Mandarin, Nepali, Punjabi, Tagalog, 
Tamil, Thai, Urdu, and Vietnamese. 


c. Includes Albanian, Bosnian, Croatian, English/Creole, French, Romanian, Russian, Turkish, and Ukrainian. 


Source: Authors’ calculations based on data from the Cleveland Metropolitan School District. 
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Table C2. Summary characteristics of schools attended by English learner students 
in grades 3-8 in the Cleveland Metropolitan School District, 2011/12-2016/17 


Characteristic 2011/12 2012/13 2013/14 2014/15 2015/16 2016/17 


Total number of schools with 


English learner students 42 42 41 44 47 41 
Average percentage of all students 
who are English learner students 10 10 10 10 10 10 
Average number of languages spoken 
among English learner students 2.8 3.0 3.2 3.1 3.4 3.8 
Average percentage of English learner students by language spoken 
Spanish 73 68 68 70 65 69 
Arabic 4 9 10 9 10 7 
School specialist staffing 
Average number of English learner 
students per certified English as 
a second language teacher? na na na na na 26 
Average number of English 
learner students per bilingual 
paraprofessional> 59 61 69 52 53 54 
Average school climate domain measures* (standard deviation) 
Academic rigor na 0.1 0.1 0.2 0.1 0.1 
Safe and respectful climate na 0.4 0.4 0.3 0.3 0.3 
Peer social-emotional learning na 0.4 0.3 0.3 0.3 0.2 
Supportive learning environment na 0.4 0.3 0.3 0.2 0.2 
Average percentage of English learner students at each English language proficiency level? 
Ohio Test of English Language Acquisition 
1—prefunctional 11. 11 15 10 na na 
2—beginning 26 33 28 30 na na 
3— intermediate 38 29 31 30 na na 
4—advanced 18 19 21 22 na na 
5—proficient 4 2 3 5 na na 
Ohio English Language Proficiency Exam 
1—emerging na na na na 13 19 
2— progressing na na na na 71 69 
3—proficient na na na na 13 11 
Average percentage of all students who met proficiency standards 
Math 49 48 50 34 26 32 
English language arts 60 60 58 47 21 29 
Percentage of schools in each school size category” 
Small 14 14 20 25 28 15 
Medium 38 50 37 36 38 51 
Large 48 36 44 39 34 34 
Average percentage of all students by race/ethnicity 
Asian 41 4 2 4 1 1 
Black 49 48 48 51 54 50 
Hispanic 22 22 22 21 19 21 
White 24 24 24 23 21 23 
Other 11 10 10 11 11 12 
(continued) 
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Table C2. Summary characteristics of schools attended by English learner students 
in grades 3-8 in the Cleveland Metropolitan School District, 2011/12-2016/17 
(continued) 


Characteristic 2011/12 2012/13 2013/14 2014/15 2015/16 2016/17 


Average percentage of all students 


with special education status 13 14 16 18 19 18 
Average percentage of all students 
with gifted status 5 5 5 5 6 7 


na is not applicable. 


a. Data on certified English as a second language teachers were not available prior to 2016/17. The average 
number of English learner students per certified English as a second language teacher was calculated only for 
schools that had at least one certified English as a second language teacher (n = 16). 


b. The average number of English learner students per bilingual paraprofessional was calculated only for 
schools that had at least one bilingual paraprofessional (n = 31 for 2011/12, n = 33 for 2012/13, n = 33 for 
2013/14, n = 42 for 2014/15, n = 33 for 2015/16, n = 32 for 2016/17). 


c. School climate data were not available for 2011/12. School climate scores were measured by aggregate 
school-level scale scores on the Conditions for Learning Survey and were standardized to the district average. 


d. The number of overall English language proficiency levels decreased from five to three between 2014/15 
and 2015/16, when the assessment changed. 


e. School size was based on the annual districtwide distribution of school enrollment. Small schools had an 
enrollment below the 25th percentile of the distribution in a given school year, medium schools had an enroll- 
ment in the 25th-75th percentiles, and large schools had an enrollment above the 75th percentile. 


Source: Authors’ calculations based on data from the Cleveland Metropolitan School District. 


Table C3. Summary characteristics of nonmultilingual schools attended by English 
learner students in grades 3-8 in the Cleveland Metropolitan School District, 


2011/12-2016/17 


Characteristic 2011/12 2012/13 2013/14 2014/15 2015/16 2016/17 


Total number of nonmultilingual 
schools that have at least one 


English learner student 32 33 32 35 38 33 
Average percentage of all students 
who are English learner students 2 3 3 3 3 4 
Percentage of nonmultilingual schools by size of English learner student population 
O-5 percent English learner 
students 91 94 81 63 71 67 
5-10 percent English learner 
students 9 6 19 37 29 33 
More than 10 percent English 
learner students 0 0) 0 0 0 0 
Average number of languages 
spoken among English learner 
students 2.1 2.2 2.6 2.4 2.6 2.9 
Average percentage of English learner students by language spoken 
Spanish 72 65 64 66 62 67 
Arabic 4 10 11 10 10 7 
School specialist staffing 
Average number of English learner 
students per certified English as 
a second language teacher? na na na na na 23 
Average number of English 
learner students per bilingual 
paraprofessional> 66 71 80 56 55 49 
Average school climate domain measures? (standard deviation) 
Academic rigor na 0.0 0.0 0.1 0.1 0.0 
Safe and respectful climate na 0.4 0.3 0.2 0.3 0.2 
Peer social-emotional learning na 0.2 0.2 0.2 0.2 0.1 
Supportive learning environment na 0.3 0.3 0.2 0.2 0.1 
Average percentage of English learner students at each English language proficiency level? 
Ohio Test of English Language Acquisition 
1—prefunctional HEE 10 16 10 na na 
2—beginning 24 33 28 30 na na 
3— intermediate 41 28 30 30 na na 
4—advanced 19 19 20 23 na na 
5—proficient 3 2 2: 4 na na 
Ohio English Language Proficiency Exam 
1—emerging na na na na 13 19 
2— progressing na na na na 71 68 
3—proficient na na na na 13 12 
Average percentage of all students who met proficiency standards 
Math 50 49 50 34 27 32 
English language arts 62 62 60 49 22 30 
(continued) 


Table C3. Summary characteristics of nonmultilingual schools attended by English 
learner students in grades 3-8 in the Cleveland Metropolitan School District, 
2011/12-2016/17 (continued) 


Characteristic 2011/12 2012/13 2013/14 2014/15 2015/16 2016/17 


Percentage of schools in each school size category® 


Small 19 18 22 29 29 15 
Medium 38 48 38 34 39 55 
Large 44 33 41 37 32 30 
Average percentage of all students by race/ethnicity 
Asian 1 1 1 1 1 1 
Black 54 55 55 57 61 57 
Hispanic 14 14 14 14 12 13 
White 26 26 26 24 22 25 
Other race/ethnicity 12 11 11 12 11 12 


Average percentage of all students 
with special education status 13 14 17 18 19 19 


Average percentage of all students 
with gifted status 5 5 5 5 6 7 


na is not applicable. 


a. Staffing data on certified English as a second language teachers were not available prior to 2016/17. The 
average number of English learner students per certified English as a second language teacher was calculated 
only for schools that had at least one certified English as a second language teacher (n = 8). 


b. The number of English learner students per bilingual paraprofessional was calculated only for schools that 
had at least one bilingual paraprofessional (n = 23 for 2011/12, n = 25 for 2012/13, n = 25 for 2013/14, 
n = 34 for 2014/15, n = 25 for 2015/16, n = 24 for 2016/17). 


c. School climate data were not available for 2011/12. School climate scores were measured by aggregate 
school-level scale scores on the Conditions for Learning Survey and were standardized to the district average. 


d. The number of overall English language proficiency levels decreased from five to three between 2014/15 
and 2015/16, when the assessment changed. 


e. School size was based on the annual districtwide distribution of school enrollment. Small schools had an 
enrollment below the 25th percentile of the distribution in a given school year, medium schools had an enroll- 
ment in the 25th—75th percentiles, and large schools had an enrollment above the 75th percentile. 


Source: Authors’ calculations based on data from the Cleveland Metropolitan School District. 


Table C4. Summary characteristics of bilingual schools attended by English 


learner students in grades 3-8 in the Cleveland Metropolitan School District, 


2011/12-2016/17 


Characteristic 2011/12 2012/13 2013/14 2014/15 2015/16 2016/17 


Total number of bilingual schools 
that have at least one English 


learner student 7 7 7 7 7 4 
Average percentage of all students 
who are English learner students 35 35 33 33 36 33 
Percentage of bilingual schools by size of English learner student population 
Less than 10 percent English 
learner students 0 0 0 0 0 0) 
10-25 percent English learner 
students 29 14 43 43 29 43 
25-40 percent English learner 
students 29 43 29 29 14 29 
More than 40 percent English 
learner students 43 43 29 29 57 29 
Average number of languages 
spoken among English learner 
students 4.1 5.3 5.0 4.9 4.6 5.4 
Average percentage of English learner students by language spoken 
Spanish 86 83 84 83 82 83 
Arabic 5 6 8 8 8 7 
School specialist staffing 
Average number of English learner 
students per certified English as a 
second language teacher na na na na na 28 
Average number of English 
learner students per bilingual 
paraprofessional’ 43 39 36 38 43 70 
Average school climate domain measures* (standard deviation) 
Academic rigor na 0.4 0.4 0.4 0.4 0.5 
Safe and respectful climate na 0.3 0.4 0.4 0.4 0.6 
Peer social-emotional learning na 0.8 0.7 0.5 0.6 0.6 
Supportive learning environment na 0.4 0.6 0.5 0.6 0.6 
Average percentage of English learner students at each English language proficiency level? 
Ohio Test of English Language Acquisition 
1—prefunctional 10 10 8 10 na na 
2—beginning 33 29 26 31 na na 
3— intermediate 33 35 37 34 na na 
4—advanced 20 22 24 19 na na 
5—proficient 4 3 5 5 na na 
Ohio English Language Proficiency Exam 
1—emerging na na na na 8 12 
2— progressing na na na na 75 78 
3—proficient na na na na 17 10 
Average percentage of all students who met proficiency standards 
Math 46 48 52 39 30 36 
English language arts 56 57 56 48 21 28 
(continued) 
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Table C4. Summary characteristics of bilingual schools attended by English learner 
students in grades 3-8 in the Cleveland Metropolitan School District, 2011/12- 
2016/17 (continued) 


Characteristic 2011/12 2012/13 2013/14 2014/15 2015/16 2016/17 


Percentage of schools in each school size category® 


Small 0 0 0 0 14 14 
Medium 29 43 43 57 43 43 
Large 71 57 57 43 43 43 
Average percentage of all students by race/ethnicity 
Asian 1 2 2 2 2 2 
Black 24 24 24 24 24 23 
Hispanic 56 56 56 55 55 55 
White 15 15 15 15 16 17 
Other 7 7 7 8 8 10 
Average percentage of all students 
with special education status 13 13 15 17 17 16 
Average percentage of all students 
with gifted status 3 2 2 3 3 6 


na is not applicable. 


a. Data on certified English as a second language teachers were not available prior to 2016/17. The average 
number of English learner students per certified English as a second language teacher was calculated only for 
schools that had at least one certified English as a second language teacher (n = 7). 


b. The number of English learner students per bilingual paraprofessional was calculated only for schools that 
had at least one bilingual paraprofessional (n = 7 for all years). 


c. School climate data were not available for 2011/12. School climate scores were measured by aggregate 
school-level scale scores on the Conditions for Learning Survey and were standardized to the district average. 


d. The number of overall English language proficiency levels decreased from five to three between 2014/15 
and 2015/16, when the assessment changed. 


e. School size was based on the annual districtwide distribution of school enrollment. Small schools had an 
enrollment below the 25th percentile of the distribution in a given school year, medium schools had an enroll- 
ment in the 25th-75th percentiles, and large schools had an enrollment above the 75th percentile. 


Source: Authors’ calculations based on data from the Cleveland Metropolitan School District. 
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Table C5. Summary characteristics of the newcomer academy attended by 
English learner students in grades 3-8 in Cleveland Metropolitan School District, 
2011/12-2016/17 


Characteristic 2011/12 2012/13 2013/14 2014/15 2015/16 2016/17 


Total number of schools with 


newcomer programs ul 1 AL 1 1 HE 
Percentage of all students who are 
English learner students 83 78 78 74 85 80 
Number of languages spoken 
among English learner students 18 12 14 17 22 21 
Percentage of English learner students by language spoken 
Spanish 54 60 61 61 58 46 
Arabic 13 8 10 13 14 18 


School specialist staffing 


Number of English learner 
students per certified English as a 


second language teacher? na na na na na 41 
Number of English learner students 

per bilingual paraprofessional® 41 33 59 52 76 63 

Average school climate domain measures (standard deviation) 

Academic rigor na 0.6 2.2 0.6 0.5 15 
Safe and respectful climate na 1.2 1.2 1.4 1.2 2.0 
Peer social-emotional learning na 2.6 2.1 1.8 1.6 2.5 
Supportive learning environment na 1.1 1.8 1.2 1.0 1.6 


Percentage of English learner students at each English language proficiency level? 


Ohio Test of English Language Acquisition 


1—prefunctional 34 51 46 37 na na 
2—beginning 36 32 38 33 na na 
3— intermediate 18 14 15 21 na na 
4—advanced 3 3 1 8 na na 
5—proficient 0 0 0 2 na na 
Ohio English Language Proficiency Exam 
1—emerging na na na na 65 70 
2— progressing na na na na 34 30 
3—proficient na na na na 1. <1 


Percentage of all students who met proficiency standards 


Math 17 9 15 15 6 4 

English language arts 10 3 6 5 1 <1 
School enrollment 399 425 520 712 949 1,064 
School size category*® Medium Medium Large Large Large Large 


Percentage of all students by race/ethnicity 


Asian 20 22 23 18 12 9 
Black 9 8 8 8 15 25 
Hispanic 59 62 57 55 56 41 
White 12 9 13 19 17 24 
Other 7 4 3 4 4 3 
Percentage of all students with 
special education status 2 1 3 6 6 4 
Percentage of all students with 
gifted status 0) 0) 0) fe) fe) 0 
(continued) 


Table C5. Summary characteristics of the newcomer academy attended by 
English learner students in grades 3-8 in Cleveland Metropolitan School District, 
2011/12-2016/17 (continued) 

na is not applicable. 

a. Data on certified English as a second language teachers were not available prior to 2016/17. 


c. School climate data were not available for 2011/12. School climate scores were measured by aggregate 
school-level scale scores on the Conditions for Learning Survey and were standardized to the district average. 


d. The number of overall English language proficiency levels decreased from five to three between 2014/15 
and 2015/16, when the assessment changed. 


e. School size was based on the annual districtwide distribution of school enrollments. Small schools had an 
enrollment below the 25th percentile of the distribution in a given school year, medium schools had an enroll- 
ment in the 25th—75th percentiles, and large schools had an enrollment above the 75th percentile. 


Source: Authors’ calculations based on data from the Cleveland Metropolitan School District. 


Table C6. Average scores on the state standardized math and English language 
arts assessments among English learner students in grades 3-8 in the Cleveland 
Metropolitan School District, 2011/12-2016/17 


English language arts 


Standard Standard 


School year CAVE LaCyn Mean CAVE Lata 
2011/12 -0.14 0.96 -0.28 0.96 
2012/13 -0.13 0.93 -0.28 0.96 
2013/14 -0.08 0.97 -0.29 0.97 
2014/15 0.04 0.92 -0.35 0.90 
2015/16 0.01 0.84 -0.11 0.84 
2016/17 -0.08 0.91 -0.24 0.87 


Note: The study team standardized math and English language arts scores relative to the distribution of 
scores across all students in the district in each year and grade level. 


Source: Authors’ analyses based on data from the Cleveland Metropolitan School District. 


Table C7. Percentage of English learner students in grades 3-8 in the Cleveland 
Metropolitan School District scoring at each level on the listening domain subscale 
of the English language proficiency assessment, 2011/12-2016/17 


Proficiency level 2011/12 2012/13 2013/14 2014/15 2015/16 2016/17 
1—prefunctional 8 10 9 9 15 17 
2—beginning 13 12 11 12 11 11 
3— intermediate 23 26 26 24 23 26 
4—advanced 19 20 20 21 29 27 
5—proficient 37 33 35 35 23 19 


Note: The English language proficiency assessment administered in 2011/12-—2014/15 was the Ohio Test 
of English Language Acquisition, and the assessment administered in 2015/16 and 2016/17 was the Ohio 
English Language Proficiency Assessment. 


Source: Authors’ analyses based on data from the Cleveland Metropolitan School District. 
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Table C8. Percentage of English learner students in grades 3-8 in the Cleveland 
Metropolitan School District scoring at each level on the speaking subscale of the 
English language proficiency assessment, 2011/12-2016/17 


Proficiency level 2011/12 2012/13 2013/14 2014/15 2015/16 2016/17 
1—prefunctional 3 5 4 4 24 28 
2—beginning 6 4 4 6 15 17 
3— intermediate 15 11 11 14 24 25 
4—advanced 31 28 24 22 18 16 
5—proficient 45 52 57 57 17 14 


Note: The English language proficiency assessment administered in 2011/12-2014/15 was the Ohio Test 
of English Language Acquisition, and the assessment administered in 2015/16 and 2016/17 was the Ohio 
English Language Proficiency Assessment. 


Source: Authors’ analyses based on data from the Cleveland Metropolitan School District. 


Table C9. Percentage of English learner students in grades 3-8 in the Cleveland 
Metropolitan School District scoring at each level on the reading subscale of the 
English language proficiency assessment, 2011/12-2016/17 


Proficiency level 2011/12 2012/13 2013/14 2014/15 2015/16 2016/17 
1—prefunctional 14 18 15 13 32 36 
2—beginning 32 33 24 34 17 19 
3— intermediate 17 17 23 20 29 29 
4—advanced 28 21 24 22 11 9 
5—proficient 10 i 13 11 10 6 


Note: The English language proficiency assessment administered in 2011/12-2014/15 was the Ohio Test 
of English Language Acquisition, and the assessment administered in 2015/16 and 2016/17 was the Ohio 
English Language Proficiency Assessment. 


Source: Authors’ analyses based on data from the Cleveland Metropolitan School District. 


Table C10. Percentage of English learner students in grades 3-8 in the Cleveland 
Metropolitan School District scoring at each level on the writing subscale of the 
English language proficiency assessment, 2011/12-2016/17 


Proficiency level 2011/12 2012/13 2013/14 2014/15 2015/16 2016/17 
1—prefunctional 20 20 22 17 24 31 
2—beginning 34 28 26 29 15 15 
3— intermediate 26 32 24 24 40 39 
4—advanced 15 19 24 19 11 9 
5—proficient 5 4 4 44 10 


Note: The English language proficiency assessment administered in 2011/12-—2014/15 was the Ohio Test 
of English Language Acquisition, and the assessment administered in 2015/16 and 2016/17 was the Ohio 
English Language Proficiency Assessment. 


Source: Authors’ analyses based on data from the Cleveland Metropolitan School District. 
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Table C11. State standardized assessment scores in math and English language 
arts regressed on student and school characteristics, 2016/17 


WEL) English language arts 


Standard Standard 
Characteristic error Beta error 


Student characteristics 
Prior-year overall English language proficiency level = 


1—emerging -0.154** 0.053 -0.275*** 0.054 
Prior-year overall English language proficiency level = 

3—proficient -0.017 0.141 0.124 0.149 
No prior-year overall English language proficiency level -0.362*** 0.062 -0.523*** 0.063 
Native language = Arabic? -0.036 0.066 0.028 0.068 
Native language = African language®” —0.108 0.060 0.027 0.061 
Native language = Asian language®° 0.058 0.070 0.113 0.071 
Native language = European/other language*4 0.105 0.102 0.098 0.105 
Female 0.020 0.033 0.023 0.033 
Special education status -0.353*** 0.047 -0.393*** 0.048 
Gifted status 0.114 0.095 0.168 0.098 
Prior-year math performance 0.731*** 0.026 na na 
No prior-year math performance 0.081 0.049 na na 
Prior-year English language arts performance na na 0.571*** 0.027 
No prior-year English language arts performance na na 0.066 0.050 


School characteristics 


Percentage of all students who are English learner 


students 0.0629 0.290 -0.577 0.443 
Percentage of English learner students whose native 

language is Spanish 0.515*** 0.114 0.278 0.172 
Number of English learner students per certified English 

as a second language teacher -0.002 0.003 -0.004 0.004 
No English as a second language teachers -0.0377 0.117 -0.237 0.186 
Number of English learner students per bilingual 

paraprofessional -0.002*** 0.000 0.000 0.001 
No bilingual paraprofessionals -0.107 0.123 -0.061 0.171 
Measure of school climate (standardized) -0.0364 0.049 0.088 0.072 
Percentage of all students who met math proficiency 

standards 0.782 0.476 -0.121 0.731 
Percentage of all students who met English language arts 

proficiency standards -0.052 0.577 0.281 0.870 
School size = small -0.313** 0.096 -0.005 0.131 
School size = medium -0.123* 0.055 -0.036 0.091 
Percentage of all students with special education status -0.806 0.481 -1.309 0.810 


* Significant at p = .05; ** significant at p = .01; *** significant at p = .001. 


na is not applicable because prior-year academic performance was used only for outcomes in the same sub- 
ject (that is, prior-year math performance was used only in the regression in which math performance was the 
outcome, and prior-year English language arts performance was used only in the regression in which English 
language arts performance was the outcome). 


Note: The study team standardized math and English language arts scores relative to the distribution of 
scores across all students in the district within each year and grade level. 


a. Compared with English learner students whose native language is Spanish. 


b. Includes Bassa, Dinka, Ewe, Igbo, Karen, Kinyamulenge, Kirundi, Krahn, Maay Maay, Somali, Swahili, 
Tigrinya, and Yoruba. 


c. Includes Burmese, Cambodian, Cantonese, Filipino, Hindi, Indonesian, Mandarin, Nepali, Punjabi, Tagalog, 
Tamil, Thai, Urdu, and Vietnamese. 


d. Includes Albanian, Bosnian, Croatian, English/Creole, French, Romanian, Russian, Turkish, and Ukrainian. 


Source: Authors’ analyses based on data from the Cleveland Metropolitan School District. 
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Aa) 


Table C12. English language proficiency levels, overall and by subscale, regressed on student- and school-level variables, 2016/17 


Listening Speaking Reading Writing 


Standard Standard Standard Standard Standard 
Characteristic error error laced error error 


Student characteristics 


Prior-year overall English language proficiency level 0.375*** 0.033 na na na na na na na na 
No prior-year overall English language proficiency level 0.262*** 0.066 na na na na na na na na 
Prior-year English language listening proficiency level na na 0.406*** 0.025 na na na na na na 
No prior-year English language listening proficiency level na na 0.085 0.108 na na na na na na 
Prior-year English language speaking proficiency level na na na na 0.407*** 0.026 na na na na 
No prior-year English language speaking proficiency level na na na na 0.088 0.101 na na na na 
Prior-year English language reading proficiency level na na na na na na 0.582*** 0.026 na na 
No prior-year English language reading proficiency level na na na na na na 0.699*** 0.090 na na 
Prior-year English language writing proficiency level na na na na na na na na 0.564*** 0.026 
No prior-year English language writing proficiency level na na na na na na na na 0.734*** 0.088 
Native language = Arabic? -0.099* 0.043 -0.299*** 0.090 -0.094 0.101 -0.208* 0.083 -0.191* 0.079 
Native language = African language®” -0.073 0.039 -0.209* 0.083 -0.011 0.094 -0.104 0.077 -0.054 0.073 
Native language = Asian language®:* -0.033 0.046 -0.160 0.097 -0.012 0.110 -0.044 0.090 -0.145 0.086 
Native language = European/other language*4 0.150* 0.064 0.212 0.135 0.316* 0.152 0.050 0.126 0.207 0.121 
Female 0.048* 0.022 0.089 0.045 0.153** 0.051 0.063 0.042 0.107** 0.040 
Special education status -0.216*** 0.031 -0.501*** 0.065 -0.530*** 0.074 -0.478*** 0.062 -0.396*** 0.059 
Gifted status 0.146* 0.063 0.154 0.134 0.074 0.149 0.227 0.125 0.205 0.118 
Prior-year math performance 0.113*** 0.018 0.250*** 0.042 0.249*** 0.046 0.153*** 0.037 0.196*** 0.035 
No prior-year math performance 0.108*** 0.033 0.375*** 0.069 0.295*** 0.076 0.039 0.063 -0.048 0.060 


School characteristics 
Percentage of all students who are English learner 


students —0.626* 0.248 -1.796*** 0.501 -2.335*** 0.577 -1.066 0.561 -0.722 0.468 
Percentage of English learner students whose native 

language is Spanish 0.189* 0.093 0.373* 0.189 0.656** 0.219 0.293 0.213 0.211 0.177 
Number of English learner students per certified English 

as a second language teacher -0.003 0.002 -0.004 0.005 -0.013* 0.005 -0.009 0.005 -0.004 0.004 
No certified English as a second language teachers -0.191 0.102 -0.375 0.206 -0.624** 0.237 -0.333 0.231 -0.224 0.192 
Number of English learner students per bilingual 

paraprofessional -0.001 0.000 -0.001 0.001 0.000 0.001 -0.001 0.001 -0.001 0.001 
No bilingual paraprofessionals -0.176 0.097 -0.344 0.199 -0.190 0.228 -0.246 0.213 -0.204 0.183 


(continued) 
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Table C12. English language proficiency levels, overall and by subscale, regressed on student- and school-level variables, 2016/17 


(continued) 
CoNY(-Y ¢-1| Listening Speaking Reading Writing 
Standard Standard Standard Standard Standard 
Characteristic error error error error error 
Measure of school climate (standardized) 0.041 0.041 0.183* 0.082 0.305** 0.096 0.088 0.091 0.031 0.077 
Percentage of all students who met math proficiency 
standards -0.618 0.404 -2.313** 0.820 -3.147*** 0.947 -1.212 0.919 -1.301 0.764 
Percentage of all students who met English 
language arts proficiency standards 0.455 0.482 2.096* 0.980 2.717* 1.130 1.160 1.094 1.656 0.911 
School size = small -0.007 0.074 -0.019 0.151 -0.133 0.175 0.096 0.166 0.073 0.142 
School size = medium -0.002 0.049 -0.005 0.100 -0.215 0.115 -0.098 0.114 -0.060 0.094 
Percentage of all students with special education status 0.587 0.451 1.382 0.911 0.811 1.052 0.079 1.032 1.615 0.855 


* Significant at p = .05; ** significant at p = .01; *** significant at p = .001. 


na is not applicable because prior-year performance was used only for the corresponding 2016/17 outcome (that is, prior-year overall English language proficiency level was used 
only in the regression in which overall proficiency level was the outcome, prior-year English language listening proficiency level was used only in the regression in which listening 
proficiency level was the outcome), and so on. 


a. Compared with English learner students whose native language is Spanish. 

b. Includes Bassa, Dinka, Ewe, Igbo, Karen, Kinyamulenge, Kirundi, Krahn, Maay Maay, Somali, Swahili, Tigrinya, and Yoruba. 

c. Includes Burmese, Cambodian, Chinese, Cantonese, Filipino, Hindi, Indonesian, Mandarin, Nepali, Punjabi, Tagalog, Tamil, Thai, Urdu, and Vietnamese. 
d. Includes Albanian, Bosnian, Croatian, English/Creole, French, Romanian, Russian, Turkish, and Ukrainian. 


Source: Authors’ analyses based on data from the Cleveland Metropolitan School District. 


Table C13. State standardized assessment scores in math and English language 
arts regressed on school climate variables, 2016/17 


Math English language arts 
School climate domain Standard error Beta Standard error 
Academic rigor 0.073 0.050 0.165** 0.061 
Safe and respectful climate -0.064 0.043 0.055 0.070 
Peer social-emotional learning —0.082 0.045 0) 0.067 
Supportive learning environment 0.055 0.056 0.147* 0.072 


* Significant at p = .05; ** significant at p = .01. 


Note: Each estimate represents a separate regression model. Each model controls for student native lan- 
guage, gender, special education status, gifted status, and prior-year math or English language arts perfor- 
mance, in addition to percentage of English learner students, percentage of English learner students whose 
native language is Spanish, number of English learner students per certified English as a second language 
teacher, number of English learner students per bilingual paraprofessional, percentage of students who met 
proficiency standards in math and English language arts in the prior year, and school size. The study team 
standardized math and English language arts scores relative to the distribution of scores across all students 
in the district within each year and grade level. 


Source: Authors’ analyses based on data from the Cleveland Metropolitan School District. 


Table C14. English language proficiency levels, overall and by subscale, regressed on school climate 
variables, 2016/17 


Listening Speaking Reading Writing 
Standard Standard Standard Standard Standard 
School climate domain error Beta error error error error 
Academic rigor 0.011 0.04 0.101 0.083 0.207* 0.098 0.036 0.087 0.024 0.074 
Safe and respectful climate 0.064 0.036 0.201** 0.074 0.305*** 0.087 0.128 0.086 0.072 0.068 


Peer social-emotional learning 0.033 0.039 0.145 0.079 0.253** 0.095 0.047 0.089 -0.016 0.075 
Supportive learning environment 0.017 0.043 0.144 0.093 0.245* 0.14 0.081 0.094 0.202 0.082 


* Significant at p = .05; ** significant at p = .01; *** significant at p = .001. 


Note: Each estimate represents a separate regression model. Each model also controls for student native language, gender, special 
education status, gifted status, and prior-year math or English language arts performance, in addition to percentage of English learner 
students, percentage of English learner students whose native language is Spanish, number of English learner students per certified 
English as a second language teacher, number of English learner students per bilingual paraprofessional, percentage of students who 
met proficiency standards in math and English language arts in the prior year, and school size. 


Source: Authors’ analyses based on data from the Cleveland Metropolitan School District. 


C-14 


Table C15. Summary of student and school characteristics related to English learner student math 
and English language arts performance and English language proficiency levels, 2016/17 


Ohio English Language 
Ohio State Test Proficiency Assessment 


English 
language 
Characteristic arts Overall Listening Speaking Reading Writing 


Student characteristics 


Prior-year English language 
proficiency level + + + + + + + 


No prior-year English language 


proficiency level - - + ns ns + + 
Native language = Arabic® ns ns ad 7 ns - - 
Native language = African language®” ns ns ns - ns ns ns 
Native language = Asian language®° ns ns ns ns ns ns ns 
Native language = European/other 

language? ns ns + ns + ns ns 
Female ns ns + ns + ns + 


Special education status - - - = = = = 


Gifted status ns ns + ns ns ns ns 
Prior-year math performance + na + + + + + 
No prior-year math performance ns na + + + ns ns 


Prior-year English language arts 
performance na + na na na na na 


No prior-year English language arts 
performance na ns na na na na na 


School characteristics 


Percentage of all students who are 
English learner students ns ns - at - ns ns 


Percentage of English learner 
students whose native language is 
Spanish + ns + + + ns ns 


Number of English learner students 
per certified English as a second 


language teacher ns ns Ns ns - ns ns 
No certified English as a second 

language teachers ns ns ns ns - ns ns 
Number of English learner students 

per bilingual paraprofessional - ns ns ns ns ns ns 
No bilingual paraprofessionals ns ns ns ns ns ns ns 


Measure of overall school climate 


(standardized) ns ns ns + + ns ns 
Academic rigor ns + ns ns + ns ns 
Safe and respectful climate ns ns ns + + ns ns 
Peer social-emotional learning ns ns ns ns + ns ns 
Supportive learning environment ns + ns ns + ns ns 
Percentage of all students who met 

math proficiency standards ns ns ns - - ns ns 
Percentage of all students who met 

English language arts proficiency 

standards ns ns ns + + ns ns 

(continued) 
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Table C15. Summary of student and school characteristics related to English learner student math 
and English language arts performance and English language proficiency levels, 2016/17 (continued) 


Ohio English Language 


Ohio State Test Proficiency Assessment 
English 
language 
Characteristic arts Overall Listening Speaking Reading Writing 
School size = small - ns ns ns ns ns ns 
School size = medium - ns ns ns ns ns ns 


Percentage of all students with 
special education status ns ns ns ns ns ns ns 


na is not applicable. 

ns is not significant at the p = .05 level. 

+ denotes a statistically significant positive relationship 

— denotes a statistically significant negative relationship 

a. Compared with English learner students whose native language is Spanish. 

b. Includes Bassa, Dinka, Ewe, Igbo, Karen, Kinyamulenge, Kirundi, Krahn, Maay Maay, Somali, Swahili, Tigrinya, and Yoruba. 


c. Includes Burmese, Cambodian, Cantonese, Filipino, Hindi, Indonesian, Mandarin, Nepali, Punjabi, Tagalog, Tamil, Thai, Urdu, and 
Vietnamese. 


d. Includes Albanian, Bosnian, Croatian, English/Creole, French, Romanian, Russian, Turkish, and Ukrainian. 


Source: Authors’ analyses based on data from the Cleveland Metropolitan School District. 
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Notes 


Across districts in REL Midwest states (Illinois, Indiana, lowa, Michigan, Minnesota, 
Ohio, and Wisconsin), the English learner student population increased 23 percent 
from 2010/11 to 2014/15, from 431,302 students to 532,042 students (U.S. Department 
of Education, National Center for Education Statistics, 2016). 

The four scales of the Conditions for Learning Survey were highly correlated. The 
study team included the scales one at a time in separate regression models to avoid 
multicollinearity. 

The proportion of all students in CMSD who are English learner students 
(5-10 percent) is comparable to that in the other largest school districts in the state, 
including Akron and Cincinnati, where English learner students account for 5 percent 
of the student population (U.S. Department of Education, National Center for Edu- 
cation Statistics, 2016). It also is similar to the proportion of English learner students 
in many other larger urban districts in the REL Midwest Region, such as Anoka- 
Hennepin, Minnesota (6 percent), and Fort Wayne, Indiana (9 percent; U.S. Depart 
ment of Education, National Center for Education Statistics, 2016). 

The one exception is that Indiana licensing requirements include a course in teaching 
English as a new language. 

These special education codes are cross-categorical, cross-categorical selfcontained, 
emotional disturbance, hearing handicap, low incidence, medically fragile, speech lan- 
guage impairment, special education prekindergarten, and visual impairment. 

The study team originally planned to standardize scores to the state means, but state 
means and standard deviations were not available for all years. 
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